[NV] Kimi-K2.6 NVFP4 B300 disagg multinode dynamo-vllm benchmarks / [NV] Kimi-K2.6 NVFP4 B300 分离式多节点 dynamo-vllm 基准测试#2181
Conversation
Add kimik2.6-fp4-b300-dynamo-vllm to configs/nvidia-master.yaml with seven 8k1k disaggregated topologies, stage the vLLM recipes under benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k2.6-fp4/8k1k/, wire the model path and a pinned srt-slurm checkout into runners/launch_b300-nv.sh (with --no-preflight for the compute-node-local model path), and append the perf-changelog entry. 中文：新增 Kimi-K2.6 NVFP4 B300 分离式多节点 dynamo-vllm 基准测试：在 configs/nvidia-master.yaml 中新增 kimik2.6-fp4-b300-dynamo-vllm（七个 8k1k 分离式拓扑），在 benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k2.6-fp4/8k1k/ 下新增 vLLM 配方文件，在 runners/launch_b300-nv.sh 中接入模型路径与固定 commit 的 srt-slurm 检出（模型路径仅存在于计算节点本地，故使用 --no-preflight），并追加 perf-changelog 记录。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
中文：将 perf-changelog 中的 pr-link 更新为 #2181。
…entry Main now requires kv-p2p-transfer on disagg entries (PR #2096); set nixl to match the recipes' NixlConnector and add the dynamo-router metadata, matching the other dynamo-vllm disagg entries. 中文：适配主分支新增的校验规则（PR #2096）：为 kimik2.6 B300 分离式配置补充 kv-p2p-transfer: nixl（与配方中的 NixlConnector 一致）及 dynamo-router 元数据， 与其他 dynamo-vllm 分离式配置保持一致。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
A shared literal engine_id across independent prefill workers collides in vLLM's decode-side NIXL handshake cache (keyed on engine_id alone), so multi-prefill topologies could silently read KV from the wrong prefill host. Omit the field so vLLM auto-generates a unique per-process id, matching every other disagg recipe in the tree. 中文：移除 kimik2.6 B300 配方中硬编码的 NixlConnector engine_id。多个独立 预填充 worker 共用同一 engine_id 时，会在 vLLM 解码侧的 NIXL 握手缓存 （仅以 engine_id 为键）中发生冲突，多预填充拓扑可能静默地从错误的预填充 节点读取 KV。省略该字段后 vLLM 会为每个进程自动生成唯一 id，与仓库中其他 分离式配方保持一致。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29291594159 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29294224072 |
srt-slurm at the pinned commit passes --device-ids to vLLM workers by default, but that flag only exists in vLLM builds newer than v0.22.0 (vllm-project/vllm#45026, merged 2026-06-20), so every worker exited with "unrecognized arguments". Set the schema's set_cuda_visible_devices fallback so srtctl binds GPUs via CUDA_VISIBLE_DEVICES instead. 中文：srt-slurm 固定 commit 默认向 vLLM worker 传递 --device-ids，但该参数 仅存在于比 v0.22.0 更新的 vLLM 版本（vllm-project/vllm#45026，2026-06-20 合入），导致所有 worker 以 "unrecognized arguments" 退出。改用 schema 提供的 set_cuda_visible_devices 回退，使 srtctl 通过 CUDA_VISIBLE_DEVICES 绑定 GPU。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29307603322 |
3 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29307603322 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29307603322 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29307603322 |
Set explicit UCX knobs for the NIXL P->D KV transfers on every worker: single-rail RMA/rendezvous, put_zcopy rendezvous scheme, memtype cache/whole-registration off (safe with cuMem-backed allocators), and CUDA-IPC-over-MNNVL off. UCX_NET_DEVICES stays cluster-provided. 中文：为 kimik2.6 B300 分离式配方补充显式 UCX 传输调优：NIXL P->D KV 传输 使用单 rail RMA/rendezvous、put_zcopy rendezvous 方案、关闭 memtype 缓存与 整块注册（对 cuMem 分配器更安全）、关闭 MNNVL 上的 CUDA-IPC。 UCX_NET_DEVICES 仍由集群环境提供。
Resolve perf-changelog.yaml: keep main's full entry list with this branch's kimik2.6-fp4-b300-dynamo-vllm entry re-appended at the end. 中文：合并 origin/main；perf-changelog.yaml 冲突按规则解决——保留 main 的 完整条目列表，并将本分支的 kimik2.6-fp4-b300-dynamo-vllm 条目重新追加到末尾。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29533443307 |
4 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29533443307 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29533443307 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29533443307 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29533443307 |
|
/reuse-sweep-run |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
functionstackx
left a comment
functionstackx
left a comment
There was a problem hiding this comment.
@adibarra has quiestion
5 participants
Description
Adds Kimi-K2.6 NVFP4 disaggregated multinode dynamo-vllm coverage on B300, parallel to the existing GB300 Kimi entries (#1861).
kimik2.6-fp4-b300-dynamo-vllmconfig inconfigs/nvidia-master.yamlwiring 8k/1k fixed-seq-len sweeps to seven recipes underrecipes/vllm/kimi-k2.6-fp4/8k1k/(dep4-dep8, dep4-tp4, dep4-tp8 topologies incl. FlashInfer-autotune variants), conc 1-4096.benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k2.6-fp4/8k1k/.runners/launch_b300-nv.shmapskimik2.6+fp4to/scratch/models/Kimi-K2.6-NVFP4, copies the recipes into a pinned srt-slurm checkout on dynamo-vllm runs, and passes--no-preflighttosrtctl applysince the model path lives on compute-node-local NVMe.perf-changelog.yamldocuments the new config key.中文说明
在 B300 上新增 Kimi-K2.6 NVFP4 分离式多节点 dynamo-vllm 基准测试，与现有 GB300 Kimi 配置（#1861）对齐。
configs/nvidia-master.yaml中新增kimik2.6-fp4-b300-dynamo-vllm配置，将 8k/1k 固定序列长度扫描接入recipes/vllm/kimi-k2.6-fp4/8k1k/下的七个配方（dep4-dep8、dep4-tp4、dep4-tp8 拓扑，含 FlashInfer autotune 变体），并发 1-4096。benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k2.6-fp4/8k1k/下新增 Slurm 配方 YAML。runners/launch_b300-nv.sh将kimik2.6+fp4映射到/scratch/models/Kimi-K2.6-NVFP4，在 dynamo-vllm 运行时将配方复制到固定 commit 的 srt-slurm 检出中，并为srtctl apply传入--no-preflight（模型路径位于计算节点本地 NVMe）。perf-changelog.yaml记录新的配置键。Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamlperf-changelog.yamlentries are appended to the end of the file (the file is chronological: oldest at top, newest at bottom)OWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR — do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps (remove and re-add the label to force one)